MEMORANDUM February 3, 2020 


TO: Wenden Sanders 
Director, Fine Arts 


FROM: Allison E. Matney, Ed.D. 
Officer, Research & Accountability 


SUBJECT: THE IMPACT OF A HOUSTON SYMPHONY PARTNERSHIP IN HISD ON 
ELEMENTARY STUDENTS’ READING AND MATHEMATICS ACHIEVEMENT, 
ATTENDANCE, AND RETENTION IN ORCHESTRA AT MIDDLE SCHOOLS, 
2019-2020 


Attached is a copy of the program evaluation conducted for the Fine Arts Department. The study 
explored the impact of students’ participation in a Houston Symphony partnership at a targeted 
HISD elementary school. Treatment groups received (1) Core Enrichment Violin Only, (2) both 
Core Enrichment and After-school Violin, or (3) After-school Violin Only from Houston 
Symphony Community-Embedded Musicians. The control group was all other students at the 
school. 


Key findings include: 

e The Houston Symphony (HS) Core Enrichment/After-school Violin group had the highest 
combined English and Spanish STAAR mean reading and mathematics scale scores among 
third- and fifth-grade students in 2019, while the HS After-school Violin Only students had 
the highest mean scale scores among fourth-grade students. 

e Paired t-test analyses showed statistically significant improvements in the mean STAAR 
mathematics scale scores of the HS Core Enrichment and the HS Core Enrichment/After- 
school treatment groups as well as the Non-HS control group as students progressed from 
third to fourth and from fourth to fifth grades (2018 to 2019). 

e The highest increases in attendance rates over the past two years as students progressed 
from third to fourth and from fourth to fifth grades were among the HS Core 
Enrichment/Afterschool Violin group. 

e Retention of the fifth-grade student cohort in orchestra at middle schools was eight percent, 
which is estimated to be below the national average. 


Further distribution of this report is at your discretion. Should you have any further questions, 
please contact me at 713-556-6700. 
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HISD 


BUREAU OF PROGRAM EVALUATION 


The Impact of a Houston Symphony Partnership in HISD on Elementary Students’ Reading and 
Mathematics Achievement, Attendance, and Retention in Orchestra at Middle Schools, 2019-2020 


Prepared by Venita R. Holmes, Dr.P.H. 


Abstract 

This study explored outcomes related to students’ participation in a Houston Symphony (HS) partnership at a targeted 
HISD elementary school. Students received weekly in-school core enrichment instruction and after-school violin 
instruction by Houston Symphony Community-Embedded Musicians throughout the 2018-2019 academic year. The 
study found that students in the HS Core Enrichment/After-school Violin group had the highest combined English and 
Spanish STAAR mean reading scale scores among third- and fifth-grade students in 2019, while the HS After-school 
Violin Only students had the highest mean scale score among fourth-grade students. In mathematics, the HS Core 
Enrichment/After-school Violin group achieved the highest mean scale score at the third and fifth-grade levels; whereas, 

the After-School Violin Only group attained the highest mean scale score at the fourth-grade level on the combined tests 
in 2019. Paired t-test analyses showed statistically significant improvements in the mean STAAR mathematics scale 
scores of the HS Core Enrichment and HS Core Enrichment/After-school groups as well as the Non-HS control group 

as students progressed from third to fourth and from fourth to fifth grades (2018 to 2019). Difference-in-differences 

(DiD) analyses revealed more consistent benefits in Houston Symphony-group participation over the control group of 
students who did not participate in the Houston Symphony. The highest increases in attendance rates over the past two 

years as students progressed from third to fourth and from fourth to fifth grades were among the HS Core Enrichment/ 
Afterschool Violin group. Retention of the fifth-grade student cohort in orchestra at middle schools was eight percent, 

which is estimated to be below the national average. Expansion of the Houston Symphony partnership to middle and 
high schools may help to improve retention outcomes for students in orchestra at these levels. 


Introduction tors collaborated to develop a quality violin instruction 

Expanding arts education in schools through com- program that will be sustainable beyond the three-year 
munity arts partnerships have been found to have lasting residency, culminating in the spring of 2021 (Figure 2). 
social and emotional benefits for students (Arts Educa- 
tion Partnership, 2018). Arts organizations have the ca- 
pacity to create dynamic and engaging lessons that foster 
student engagement in schools. Moreover, after-school 
learning experiences offered through arts organizations 
have the potential to reinforce in-school learning and en- 
hance academic achievement (Jones, 2018). The benefits 
of arts partnerships is related to the quality of arts experi- 
ences provided to students, which should be suitable and 
relevant to their individual needs and incorporate best 
practices in learning, pedagogy, community dynamics, 
and environment (Bamford, 2010). 

To that end, the Houston Independent School District’s 
(HISD) Fine Arts Department provides many opportuni- 
ties for students to develop as emerging artists through 
partnerships with local, state, and national organizations 


: : ; Figure 1: Lewis Elementary after-school violin class performs 
(Figure 1). Through a partnership with the Houston _ jn Jones Hall lobby before a Houston Symphony Family Con- 
Symphony, elementary school teachers and administra- cert with community-embedded musician 
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Figure 2: HISD Fine Arts Department/Houston Symphony partmership 
logic model 


To compliment core enrichment classes, the Houston Sym- 
phony provides third through fifth-grade music students weekly 
in-school and after-school violin classes in a targeted elementary 
school (Figure 3 and Figure 4). The Houston Symphony offers 
the broader student community and family members access to the 
Symphony and its music through its Student Concerts and Fam- 
ily Concerts Series (Figure 5). The Symphony will expand the 
partnership with HISD to ensure student graduates have access 
to high-quality string instruction programs in the Chavez High 
School feeder pattern if they choose to continue musical studies in 
middle and high schools. 


Background 

Fine arts teaching strategies in HISD are designed to provide 
students with high-quality arts educational experiences through 
observation and perception, creative expression, historical and cul- 
tural relevance, and critical evaluation and response (HISD Fine 
Arts Department, 2019; Texas Education Agency, 2016). Students 
are empowered to become self-motivated, adaptable, productive, 
competent, and lifelong learners who are equipped to meet the 
demands of the global workforce. 

A community arts partnership with the Houston Symphony 
may play a pivotal role toward enhancing students’ arts experi- 
ences. However, to ensure that the partnership is successful, class- 
room teachers and Community-Embedded Musicians must work 
jointly to develop curriculum and have ongoing, relevant profes- 
sional development. Collaborative work should support the State 
of Texas curriculum and fine arts teaching strategies to build on 
student learning and improve student outcomes. This evaluation 
was designed to explore the impact of the Houston Symphony 
partnership toward meeting student outcomes in a targeted HISD 
elementary school. The research questions addressed in the evalu- 
ation are as follows. 


Research Questions: 

1. What were the demographic characteristics of students who 
participated in the Houston Symphony partnership at the targeted 
HISD elementary school? 

2. How did students who received direct instruction from Hous- 
ton Symphony Community-Embedded musicians perform on the 
STAAR reading and mathematics assessments relative to con- 
trol-group students in 2018-2019? 
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Figure 3: Houston Symphony Community-Embedded Musician demon- 
strates to students how to bring the violin to playing position 


3. To what extent did the reading and mathematics performance 
of Houston Symphony partnership students and control-group stu- 
dents improve over the past two years? 

4. How did school attendance of Houston Symphony partnership 
students compare to the attendance of students who did not partic- 
ipate in the partnership? 

5. What proportion of the fifth-grade Houston Symphony student 
cohort were retained in orchestra at HISD middle schools during 
the 2019-2020 academic year? 


Review of the Literature 

The National Endowment for the Arts (2012) maintains that 
“At-risk students who have access to the arts tend to have better 
academic results, better workforce opportunities, and more civic 
engagement” (p. 1). Students not offered education in the arts lose 
an opportunity to experience a wider array of cognitive, social, and 
emotional dispositions that the arts may promote (Gadsden, 2008). 

There have been numerous research studies that examined the 
relationship between arts participation and academic achievement 
among youth. Gillespie, Russell, and Hamann (2014) found that 
new string programs had several benefits, including increased stu- 
dent participation in all music programs, more student collabora- 


Figure 4: Third-grade students learn the five steps of getting into playing 
position 


Figure 5: After-school violin class HISD students with Houston Symphony community-embedded musicians following lobby concert at Jones Hall 


tive opportunities, increased community support by music busi- 
nesses, and more comprehensive music curriculum. 

Hetland (2000) reports positive effects of music listening and 
spatial reasoning as well as music instruction and spatial rea- 
soning among children from economically-disadvantaged back- 
grounds. Academic benefits for students have been observed in 
content-area courses (Deasy, 2002; Hattie, 2009), with improve- 
ments in concentration, intrinsic motivation (Shernoff & Vandell, 
2007), problem solving skills (Catterall, 2007), and educational 
aspirations (Marsh & Kleitman, 2002). Non-academic benefits 
of arts participation were found to be associated with enhanced 
self-worth (Blomfield & Barber, 2011), empathy (Hunter, 2005), 
well-being, healthy social relationships (Rose-Krasnor, Busseri, 
Willoughby, & Chalmers, 2006), leadership skills (Hancock, Dyk, 
& Jones, 2012), and reduced risky behavior (Miller et al., 1998). 

Forgeard, Winner, Norton, and Schlaug (2008) noted that 
instrumental music training may enhance auditory discrimina- 
tion, vocabulary, and non-verbal reasoning skills among youth. 
The study included 59 public school children, 8 to 12-years old, 
who received at least three years of musical training and a con- 
trol group that received no musical training. Children who re- 
ceived musical training outperformed control-group children in 
language development. Moreover, the longer children spent in 
musical training, the more likely they were to outperform children 
who did not receive training. Southgate and Roscigno (2009) in- 
vestigated the association between music training and academic 
achievement among 13-17 years old children who attended in- 
and out-of-school activities. Involvement in music had a positive 
association on students’ mathematics and reading scores as well 
as course grades. 

Community arts partnership have the capacity to bring a 
wealth of skills and resources to schools and bridge the gap in 
student learning and outcomes (National Guild for Community 
Arts Education, 2015). Arts partnerships have been found to sup- 
port student engagement, student voice, social learning, creative 
skills, and arts-related knowledge and skills (Imms, Jeanneret, & 
Stevens-Ballenger, 2011). Scripp, Burnaford, Vazquez, Paradis, 
and Sienkiewicz (2013) found that teachers who participated in 
a partnership “excelled at arts integration curriculum design, the 
ability to articulate arts integration program goals and outcomes, 
effective collaboration with teaching artists, and the documenta- 
tion and organization of student work portfolios” (p. 1). However, 
retention of elementary school students in instrumental music pro- 
grams is of concern to music educators (Cook, 2013). Persistent 
attrition of students could lead to fewer artistically-oriented citi- 
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zens to appreciate, understand, teach, and play good music (Kruth, 
1964). Strengthening community arts partnerships at all levels of 
school may have long-term benefits for students and the commu- 
nity at large. 


Methods 

This quasi-experimental study explored whether instruction 
provided by Houston Symphony Community-Embedded Musi- 
cians influenced the reading and mathematics performance, school 
attendance, and retention of students in orchestra at middle school. 
The methodology used to identify the study population, control 
group, and data sources are described below. 


Study Population 

To reduce the threats to validity, treatment and control student- 
groups were established. Treatment groups received (1) Core En- 
richment Violin Only, (2) both Core Enrichment and After-school 
Violin, or (3) After-school Violin Only from Houston Symphony 
Community-Embedded Musicians. The control group was all oth- 
er students at the targeted elementary school who did not receive 
instruction through the musicians (Non-Houston Symphony Stu- 
dents). To determine the treatment groups, teacher-student linkag- 
es were made using class rosters extracted from the IBM Cognos 
data system on November 10, 2019. 


Data Collection and Analyses 

The HISD Chancery database was used to gather demograph- 
ic characteristics of the study population and the control group. 
Academic achievement data were obtained from (STAAR) data 
files (June 27, 2019). Reading and mathematics performance was 
assessed, considering the preponderance of research that links per- 
formance in these areas to student success (Espin & Deno, 1993; 
Duncan et al., 2007; Balfanz, Herzog, & Mac Iver, 2007; Kena et 
al., 2016). 

Descriptive statistics were calculated to determine the percent- 
age of students at or above the Approaches Grade Level passing 
standard on the first administration of the combined English and 
Spanish STAAR Grades 3-8 examinations. According to the Tex- 
as Education Agency (2017), a student achieving the Approach- 
es Grade Level standard is likely to succeed in the next grade 
or course with targeted academic intervention. Students in this 
category typically demonstrate the ability to apply the assessed 
knowledge and skills in familiar contexts (Texas Education Agen- 
cy, 2017). 


Paired samples t-tests were conducted using scale scores from 
the first administration of the combined reading and mathematics 
STAAR 3-8 for students with test data in 2017-2018 and 2018-— 
2019. The level of statistical significance was p < .05, two-tailed 
test. The one-tailed t-test was also considered in the analyses to 
detect statistical significance. The Difference-in-Differences (DiD) 
technique was used to obtain an appropriate counterfactual to es- 
timate a causal effect of the program on the paired reading and 
mathematics scores between treatment groups and the control 
group (Bertrand, Duflo, & Mullainathan, 2004; Zhou, Taber, Ar- 
cona, & Li, 2016). Zhou et al. (2016) demonstrated that DiD can 
be applied to estimate treatment effects in a heterogeneous popula- 
tion, where the treatment and control groups varied greatly. “DiD 
offers a robust method for comparing diverse cohorts when other 
risk-adjustment methods may not be adequate” (Zhou et al., 2016, 
p. 414). 

Attendance data for the 2017-2018 (pretest) and the 2018— 
2019 (posttest) academic years were extracted from the Cognos 
data system on November 10, 2019 to measure change in students’ 
attendance over time. These data included the number of days 
present and the total number of days in attendance. Only students 
with data in both years were included in the analyses. 

Fifth-grade Houston Symphony treatment group students were 
tracked to determine whether they enrolled in orchestra classes in 
middle school. The retention rate was calculated based on the total 
number of fifth grade students in the cohort enrolled in an HISD 
school during the 2019-2020 school year and the number of stu- 
dents enrolled in orchestra at sixth grade. 


Study Limitations 

There were several limitations of the study. Specifically, the 
study population only included students at one HISD elementary 
school where students were provided direct instruction by Houston 
Symphony Community-Embedded Musicians during the 2018— 
2019 year. The study assumed that rosters with student group 
assignments were accurate and that students in the control group 
did not receive instruction by the musicians. While the demo- 
graphic characteristics of students in each group may have varied, 
the groups shared a similar educational environment at the same 
school. Using a control group who attended the same school as the 
treatment group helped to mitigate some limitations to generate 
more valid treatment and comparison group analyses. In addition, 
statistical methods were used to establish equivalence between the 
groups by controlling for previous years’ performance. The study 
did not control for whether or not students had multiple years of 
music instruction through the Houston Symphony partnership. Stu- 
dent outcome measures were limited to the availability of the data 
in HISD data systems; therefore, students who lacked data on vari- 
ables of interest were excluded from the study. Other factors may 
have greatly influenced students’ educational outcomes, including 
the quality of instruction provided by the musicians (Camilli et al., 
2010), student engagement in music classes, and whether students 
were, purposefully, selected to participate in classes with Houston 
Symphony instructors. 


Results 


What were the demographic characteristics of students who 
participated in the Houston Symphony partnership at the tar- 
geted HISD elementary school? 
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The study population consisted of third through fifth-grade 
music students who participated in both in-school core enrich- 
ment and after-school violin classes weekly at an HISD elementa- 
ry school during the 2018-2019 academic year. The name of the 
HISD teacher whose students received instruction from Houston 
Symphony Community-Embedded Musicians was provided by 
HISD Fine Arts Department staff. A control group was established 
consisting of all students in grades three through five who did not 
participate in the partnership at the same school. Profiles of treat- 
ment groups and control-group students are presented in Table 1 
(Appendix A, p. 10). 

In summary, there were 292 students in the Non-Houston 
Symphony (HS) control-group and 209 students in the Houston 
Symphony (HS) treatment group. The HS treatment groups were 
comprised of Core Enrichment Violin Only (n=175), Core Enrich- 
ment/After-school Violin (n=29), and After-school Violin Only 
(n=5) students. 

Students in all groups, except the HS After-school Violin Only 
group, were mostly Hispanic. This group was predominately A fri- 
can American. The HS Core Enrichment/After-school Violin and 
the Non-HS Student group were more likely to be female than 
male. The majority of students in all groups were economically 
disadvantaged and none of the students in the groups were cate- 
gorized as at risk of dropping out of school. HS Core Enrichment/ 
After-school Violin and HS After-school Violin Only groups had 
much higher percentages of gifted/talented (G/T) students than the 
Non-HS student group. While the HS After-school Violin Only 
group had a much lower percentage of limited English proficient 
(LEP) students than all other groups, the Non-HS group had lower 
percentages of LEP students than the HS Core Enrichment Violin 
Only and the HS Core Enrichment/After-school Violin groups. 


How did students who received instruction from Houston 
Symphony Community-Embedded Musicians perform on the 
STAAR reading and mathematics assessments relative to con- 
trol-group students in 2018-2019? 


The percent of students who scored At or Above Approach- 
es Grade Level standard (met standard) on the combined English 
and Spanish STAAR 3-8 reading assessment for the 2018-2019 
academic year can be found in Figure 6 and in Table 2a (Appen- 
dix B, p. 11). Findings related to the HS After-school Violin Only 
group should be viewed with caution considering the small sam- 
ple size. The findings revealed that all third-grade students in the 
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Figure 6: Combined English and Spanish reading STAAR by grade level, percent 
met Approaches Grade Level standard, 2018-2019 
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Figure 7: Combined English and Spanish reading STAAR by grade level, mean 
scale scores, 2018-2019 


HS Core Enrichment/After-school Violin group as well as the HS 
After-school Violin Only group met the passing standard on the 
reading assessment. Among fourth-graders, the HS After-school 
Violin Only student group had the highest passing rate; while the 
HS Core Enrichment/After-school Violin students had the highest 
passing rate among fifth-graders. It should be noted that findings 
related to the HS After-school Violin Only students were not rep- 
resented in the fifth-grade group. 

Figure 7 and Table 2b (Appendix B, p. 12) depicts the mean 
reading scale scores for the student groups. The HS Core Enrich- 
ment/After-school Violin group had the highest mean scale score 
among third-grade and fifth-grade students, while the HS Af- 
ter-school Violin Only students had the highest mean scale score 
among fourth-grade students. 

Figure 8 and Table 3a (Appendix C, p. 13) shows mathemat- 
ics STAAR results by student group and grade level. The HS Af- 
ter-school Violin Only group had the highest passing rate among 
the third and fourth-grade students; whereas, the HS Core En- 
richment/After-school Violin group had the highest passing rate 
among fifth-grade students. After-school Violin Only students 
were not represented among fifth graders. In addition, group data 
with After-school Violin Only students should be viewed with cau- 
tion, considering the small number of students in the group. 

Figure 9 and Table 3b (Appendix C, p. 14) depicts the mean 
scale scores of the student groups on the STAAR mathematics 
assessment. The HS Core Enrichment/After-school Violin group 
achieved the highest mean scale score at the third and fifth-grade 
levels and the After-School Violin Only group attained the highest 
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Figure 8: Combined English and Spanish mathematics STAAR by grade level, 
mean scale score, 2018-2019 
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Figure 9: Combined English and Spanish mathematics STAAR by grade level, 
mean scale score, 2018-2019 


mean scale score at the fourth-grade level. There were no fifth- 
grade After-school Only students in the data. 


To what extent did the reading and mathematics performance 
of Houston Symphony partnership students and control-group 
students improve over the past two years? 


A paired t-test was conducted to determine the extent that stu- 
dents who received instruction from Houston Symphony Com- 
munity-Embedded Musicians improved their reading and math- 
ematics performance from the 2017-2018 academic year to the 
2018-2019 academic year. The results of students with two years 
of STAAR 3-8 reading and mathematics data (2018 and 2019) in 
English or Spanish over the two-year period were used in the anal- 
yses. Scale score comparisons reflected successive progression in 
grade level testing from year-to-year. 

Figure 10 and Appendix 4a (p. 15) reveals statistically sig- 
nificant increases in the mean reading scale scores of the HS Core 
Enrichment Violin Only group and the Non-HS Students as they 
progressed from third to fourth grades (p < .001). The largest in- 
crease was among the HS Core Enrichment Violin Only treatment 
group (72 points), followed by the Non-HS control student group 
(65 points). 

Figure 11 (Appendix 4a, p. 15) shows statistically significant 
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Figure 10: Paired T-test analyses, students with two years of combined English 
and Spanish reading STAAR, successive progression from third (2018) to fourth 
(2019) grades 


increases in the mean reading scale scores of the HS Core En- 
richment Violin Only, the HS Core Enrichment/After-school, and 
the Non-HS student groups as they progressed from fourth to fifth 
grades (p < .001). The largest increase was among the HS Core 
Enrichment/A fter-school Violin group (161 points). 

Paired mathematics results of the third to fourth-grade student 
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Figure 11: Paired T-test analyses, students with two years of combined English 
and Spanish reading STAAR, successive progression from fourth (2018) to fifth 
(2019) grades 


groups are shown in Figure 12 and Table 4b (p. 15). Statistically 
significant increases were found among the HS Core Enrichment 
Violin Only, the HS Core Enrichment/After-school and the Non- 
HS student groups. However, the Non-HS control group showed 
the most gain over the two-year period (128 points). Among the 
treatment groups, HS After-school Violin Only students experi- 
enced the most gain of 109 points, closely followed by HS Core 
Enrichment Violin Only students (106 points). 
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Figure 12: Paired T-test analyses, students with two years of combined English 
and Spanish mathematics STAAR, successive progression from third (2018) to 
fourth (2019) grades 


Paired mathematics results of fourth to fifth-grade student 
groups are shown in Figure 13. Statistically significant increases 
were found among the HS Core Enrichment Violin Only, the HS 
Core Enrichment/After-school (one-tailed test), and the Non-HS 
Student groups. However, HS Core Enrichment/After-school Vi- 
olin students made the largest gain of 122 points. It is also evident 
that HS Core Enrichment Violin Only students experienced an 
increase of 65 points; whereas, Non-HS Students had an increase 
of 58 points from 2018 to 2019. 

Difference-in-differences (DiD) analyses (Figure 14) revea- 
led a benefit in treatment-group participation over the control 
group as evidenced by a higher mean reading score of 7 points as 
students progressed from third to fourth grade. Moreover, as stu- 
dents progressed from fourth to fifth grades, there were benefits 
observed in HS Core Enrichment Violin Only participation by 10 
points and in HS Core Enrichment/After-school Violin participa- 
tion by 105 points over Non-HS participation. 

Difference-in-Differences (DiD) analyses revealed a benefit 
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Figure 13: Paired T-test analyses, students with two years of combined English 
and Spanish mathematics STAAR, successive progression from fourth (2018) to 
fifth (2019) grades 
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Figure 14: Difference-in-Difference estimations, HS treated groups vs. Non-HS 
group using 2018 and 2019 paired STAAR reading data 


in HS Core Enrichment Violin Only participation over the con- 
trol group as evidenced by a higher mean mathematics score of 
7 points as students progressed from fourth to fifth grade (Fig- 
ure 15). A benefit was also observed in HS Core Enrichment/ 
After-school Violin participation by 64 points over Non-HS par- 
ticipation as students progressed from fourth to fifth grade. 
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Figure 15: Difference-in-Differences estimations, HS treated groups vs. Non-HS 
group using 2018 and 2019 paired STAAR mathematics data 
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Figure 16: Attendance rates by Houston Symphony and Non-Houston Symphony 
student groups, 2018-2019 


How did school attendance of Houston Symphony partnership 
students compare to the attendance of students who did not 
participate in the partnership? 


The mean attendance rates for the Houston Symphony part- 
nership student groups and the control group for the 2018-2019 
academic year are displayed in Figure 16 and in Appendix E, p. 
16. It is apparent that the highest attendance rate was among HS 
Core Enrichment/After-school Violin students (96.8 percent). The 
Non-HS Students had an attendance rate of 95.5 percent. HS Af- 
ter-school Violin only students had the lowest attendance rate. This 
finding should be viewed with caution considering the small sam- 
ple size. 

The 2018-2019 attendance data were used as a posttest, and 
the 2017-2018 attendance data were used as a pretest to estimate 
change in attendance over time. Only students with two years of 
data were included in the analyses. Figure 17 shows increases in 
the mean attendance rates of all student groups as they progressed 
from third grade to fourth grade. The highest increase was among 
the HS Core Enrichment/Afterschool Violin group (95.9 percent 
vs. 97.3 percent or 1.4 percentage points). The attendance rate of 
the Non-HS Students increased by one percentage point over the 
two-year period. 

Figure 18 shows the mean attendance rates of HS treatment 
groups and Non-HS Students as they progressed from fourth to 
fifth grades. Insufficient data were available to detect changes in 
attendance rates for the After-school Violin Only group. It is evi- 
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Figure 17: Paired attendance rates by Houston Symphony and Non-Houston 
Symphony student groups, grade 3 (2017-2018) to grade 4 (2018-2019) 


HISD Department of Research and Accountability 


Paired Attendance, Grade 4 to Grade 5 
99.0 


98.5 98.3 

98.0 
g 97.5 
2 

97.0 
2 ee ae 96.5 

96.5 
E SS SS 96.2 96.0 
96.0 NS \S 

95.0 N N N 

wes SS S S 

HS Core Enrichment Violin HS Core Enrichment/After- Non-HS Students 
Only school Violin 


~ Grade 4 (2017-2018) | ™ Grade 5(2018-2019) 


Figure 18: Paired attendance rates by Houston Symphony and Non-Houston 
Symphony student groups, grade 4 (2017-2018) to grade 5 (2018-2019) 


dent that only students in the HS Core Enrichment/A fter-school 
Violin group showed an increase in the mean attendance rate 
from 2017-2018 (96.5 percent) to 2018-2019 (98.3 percent). 
The difference was by 1.8 percentage points. 


What proportion of the fifth-grade Houston Symphony 
student cohort were retained in orchestra at HISD middle 
schools during the 2019-2020 academic year? 


Several studies have examined factors that may contribute 
to students’ retention and attrition in instrumental music edu- 
cation at postsecondary levels (Hamann & Gillespie, 1998; 
Kuhlman, 2005; Witt & Goodrich, 2003). Factors identified in 
the research include “(1) a new teacher when changing schools, 
(2) socioeconomic status, (3) motivational reasons (including 
family, peer, and self), (4) scholastic achievement, (5) the grade 
level in which instruction begins, and (6) scheduling conflicts” 
(Cook, 2013, p. 3). Hamann and Gillespie (2002) estimated that 
the average national orchestra student attrition rate is 27% be- 
tween elementary and middle school. 

To determine the influence of the Houston Symphony part- 
nership on retention in orchestra among the study population, 
the 2018-2019 fifth-grade student cohort was tracked to sixth 
grade. The results are depicted in Figure 19. It was evident that 
63 fifth-grade students from the cohort continued enrollment 
in HISD middle schools during the 2019-2020 academic year. 
Five of the students were enrolled in an orchestra course in 
sixth grade, yielding an 8 percent retention rate. These students 
attended three middle schools in the east area of HISD. 
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Figure 19: Retention of 5th-grade student cohort in HISD middle schools, 
2019-2020 


Discussion 

The HISD Fine Arts Department emphasizes the importance 
of community partnerships to support a quality fine arts education 
and the development of the whole child. Consistent exposure to 
fine arts has been found to enhance student engagement, which 
enhances academic achievement. Students have expanded oppor- 
tunities to develop their abilities and habits of mind that empower 
them to learn across multiple content areas. 

This evaluation explored the impact of the Houston Sympho- 
ny partnership at a targeted HISD elementary school on student 
achievement, attendance, and retention in orchestra. The study 
used a retrospective design, and students were not randomized 
to treatment or control groups. An underlying assumption of the 
study was that the partnership provided quality arts instruction 
to students that was aligned to state standards. Treatment groups 
were established based on Houston Symphony involvement and 
a control-group consisting of students with similar background 
characteristics at the same school strengthened the validity of the 
study. 

Paired t-test analyses showed statistically significant improve- 
ments in the mean STAAR reading and mathematics scale scores 
of HS Core Enrichment Violin Only, HS Core Enrichment/Af- 
ter-school Violin, After-school Only and Non-HS Student groups 
as they progressed from third to fourth and from fourth to fifth 
grades. The analyses was conducted using 2018 and 2019 test 
data. Difference-in-differences (DiD) analyses revealed more con- 
sistent benefits in Houston Symphony-group participation over 
the control group that did not participate in the Houston Sympho- 
ny for HS Core Enrichment Violin Only students and HS Core 
Enrichment/After-school Violin students as evidenced by high- 
er mean reading and mathematics scores as students progressed 
from third to fourth and from fourth to fifth grade. Based on the 
research, retention of fifth-grade cohort students in orchestra at 
middle school was eight percent, which is estimated to be below 
the national average (Hamann & Gillespie, 2002). 

In consideration of study findings, there was evidence of ben- 
efits of the Houston Symphony partnership toward improving stu- 
dents’ academic performance and school attendance. Evidence of 
the partnerships’ ability to retain students in orchestra at middle 
school was not substantiated. To ensure that the partnership re- 
mains successful and has lasting effects, classroom teachers and 
teaching artists must work jointly to develop curriculum that is 
aligned to state standards. Expanding the partnership beyond the 
elementary level to middle and high school students may help to 
promote retention of students in orchestra. 
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Appendix A 


Table 1: Demographic Characteristics of Houston Symphony (HS) Violin Treatment Group Students and Non-Houston Symphony Control 


Group Students (Grades 3 through 5), 2018-2019 


Treatment Groups (N=209) 
HS Core HS Core 
Enrichment Enrichment/ HS After-school Non-Houston Symphony 
Violin Only After-school Violin Violin Only Control Group 
(N= 175) (N=29) (N=5) (N=292) 

Race/Ethnicity N % N % N % N % 

American Indian 0 - 0 - 0 - - - 

Black 28 16.0 4 13.8 3 60.0 72 24.7 

Hispanic 141 80.6 24 82.8 2 40.0 216 74.0 

White 3 1.7 0 - 0 - 1 3 

Two or More Races 3 1.7 1 3.4 0 - 2 7 
Gender 

Male 109 62.3 13 44.8 3 60.0 112 38.4 

Female 66 37.7 16 55:2 2 40.0 180 61.6 
Eco Disadv. 171 97.7 27 93.1 5 100.0 288 98.6 
At Risk 0 - 0 - 0 - 0 - 
Special Ed 19 10.9 0 - 1 20.0 1 11.3 
G/T als 9.7 6 20.7 1 20.0 21 7.2 
LEP 116 66.3 19 65.5 1 0.2 170 58.2 
Source: Chancery and PEIMS databases 
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Appendix B 


Table 2a: STAAR Combined English and Spanish Reading Percent At or Approaches Grade Level Standard, 2018-2019 


Grade Level 


At or Above Approaches 
Grade Level Standard 


Third Did Not Meet 22 40.7 
Met 32 59.3 
Total 54 100.0 
Fourth Did Not Meet 32 57.1 
Met 24 42.9 
Total 56 100.0 
Fifth Did Not Meet 27 43.5 
Met 35 56.5 
Total 62 100.0 


Third Met 10 100.0 
Fourth Did Not Meet 6 42.9 
Met S701 
Total 14 100.0 
Fifth Met 7 100.0 


Third Did Not Meet 22 26.2 
Met 62 73.8 
Total 84 100.0 

Fourth Did Not Meet 42 44.7 
Met 52 55:3 
Total 94 100.0 

Fifth Did Not Meet 28 31.5 
Met 61 68.5 
Total 89 100.0 


*Less than 5 students 
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Grade Level 


Appendix B (cont’d) 


Table 2b: STAAR Combined English and Spanish Reading Mean Scale Score, 2018-2019 


Minimum 


Maximum 


Mean 


Std. Deviation 


Third 54 1216 1812 1393.39 144,226 
Fourth 56 1150 1766 1423.59 132.859 
Fifth 62 1233 1870 1516.90 147.519 


Third 10 1356 1812 1549.60 140.957 
Fourth 14 1283 1715 1456.00 111.335 
Fifth 7 1511 1870 1694.14 145.011 
Third 1 * il a ii 
Fourth 3 id = * = 


Third 84 993 1812 1439.92 164.126 
Fourth 94 1083 1851 1438.28 137.115 
Fifth 89 1255 1787 1509.10 124.641 


*Less than 5 students 
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Table 3a: STAAR Combined English and Spanish Mathematics Percent At or Approaches Grade Level Standard, 2018-2019 


Appendix C 


Grade Level 


At or Above Approaches 
Grade Level Standard 


Third Did Not Meet 16 29.6 
Met 38 70.4 
Total 54 100.0 
Fourth Did Not Meet 21 37.5 
Met 35 62.5 
Total 56 100.0 
Fifth Did Not Meet 17 27.4 
Met 45 72.6 
Total 62 100.0 


Third Did Not Meet 1 . 
Met 9 90.0 
Total 10 100.0 

Fourth Did Not Meet 6 42.9 
Met By 
Total 14 100.0 

Fifth Did Not Meet 1 ad 
Met 6 85.7 
Total 7 100.0 


Third Met 1 i 
Fourth Met 3 = 
[Now-Finesstudents ff 
Third Did Not Meet 15 17.9 
Met 69 82.1 
Total 84 100.0 
Fourth Did Not Meet 29 30.9 
Met 65 69.1 
Total 94 100.0 
Fifth Did Not Meet Al 45.6 
Met 49 54.4 
Total 90 100.0 


*Less than 5 students 
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Table 3b: STAAR Combined English and Spanish Mathematics Mean Scale Score, 2018-2019 


Appendix C (cont’d) 


Grade Level 


Minimum 


Mean 


Std. Deviation 


Third 54 1186 1897 1425.65 121.150 
Fourth 56 1274 1770 1505.61 130.597 
Fifth 62 1308 2078 1593.76 170.128 


Third 10 1333 1762 1515.20 119.395 
Fourth 14 1395 1700 1519.29 99.612 
Fifth Z 1308 2078 1706.71 250.070 
Third 1 * s > = 
Fourth 3 * - ? 


Third 84 1105 1897 1456.26 129.035 
Fourth 94 1214 2040 1532.76 163.337 
Fifth 90 1218 1860 1540.62 157.535 


*Less than 5 students 
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Appendix D 


Table 4a: Combined English and Spanish Reading STAAR 3-8 Paired T-test Analyses, Houston Symphony Treatment Student Groups and 


Control Student Group with 2 Years of Data, Successive Progression from Third to Fourth Grades and Fourth to Fifth Grades, 2018 and 2019 
(First Test Administration) 


Reading STAAR 
2018 2019 Std. Std. 
Student Groups Mean Scale Mean Scale Mean Deviation | Deviation 
Score Score Diff. N 2017 2018 t Sig. 
Grade 3 Grade 4 
HS Core Enrichment Violin 1356 1428 72 43 130.2817 138.751 4.730 .000** 
Only 
HS Core Enrichment/ 1422 1454 32 55 139.840 155.921 .907 384 
After-school 
HS After-school Only ++ ++ ++ 2 ++ ++ +4 +H 
Non-HS Students 1388 1453 65 65 124.4197 137.994 6.269 .000*** 
Grade 4 Grade 5 
HS Core Enrichment Violin 1444 1510 66 55 139.8400 155.921 4.827 .000*** 
Only 
HS Core Enrichment/ 1580 1741 161 5 153.7573 140.389 4.501 .000*** 
After-school 
HS After-school Only - - - - - - - - 
Non-HS Students 1461 1517 56 65 127.7081 129.611 3.825 .000*** 


Source: Cognos database for 2018-2019 clsss rosters to link teachers with students; Combined English and Spanish STAAR 3-8 database, spring 2018 and 
spring 2019 paired data 

*p< .05 

**p<.01 

*< 001 

++Less than 5 students 


Table 4b: Combined English and Spanish Mathematics STAAR 3-8 Paired T-test Analyses, Houston Symphony Treatment Student Groups 


and Control Student Group with 2 Years of Data, Successive Progression from Third to Fourth Grades and Fourth to Fifth Grades, 2018 and 
2019 (First Test Administration) 


Mathematics STAAR 
2018 2019 Std. Std. 
Student Groups Mean Scale Mean Scale Mean Deviation | Deviation 
Score Score Diff. N 2017 2018 t Sig. 
Grade 3 Grade 4 
HS Core Enrichment Violin 1401 1507 106 43 96.2615 127.790 7.665 .000*** 
Only 
HS Core Enrichment/ 1445 1507 62 12 88.8108 100.019 3.137 .009** 
After-school 
HS After-school Only ++ ++ ++ 2 ++ ++ ++ +H 
Non-HS Students 1435 1563 128 65 115.2918 158.683 11.026 .000*** 
Grade 4 Grade 5 
HS Core Enrichment Violin 1532 1597 65 55 145.4412 175.098 4.655 .000*** 
Only 
HS Core Enrichment/ 1693 1815 122 5 153.5757 184.891 2.398 .075+ 
After-school 
HS After-school Only - - - - = = = = 
Non-HS Students 1519 1577 58 65 145.3702 152.205 4.942 .000*** 


Source: Cognos database for 2018-2019 clsss rosters to link teachers with students; Combined English and Spanish STAAR 3-8 database, spring 2018 and 
spring 2019 paired data 

*p< .05 

*D< O01 

**D<.001 

+statistically significant, one-tailed test at p<.05 

++Less than 5 students 
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Table 5: Attendance, 2018-2019 


Appendix E 


Attendance, 2018-2019 


Student Groups N Minimun Maximum Mean Std. Deviation 
HS Core Enrichment Violin Only 175 80.4 100.0 96.053 4.1784 
HS Core Enrichment/A fter-school Violin 29 84.7 100.0 96.772 3.7354 
HS After-school Violin Only 5 73.8 100.0 91.137 10.4001 
Non-HS Comparison 292 0 100.0 95.511 7.7706 
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